1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method in which a multi-band image taken with a multi-band camera is color-separated for output to an image output apparatus.
2. Description of the Related Art
Digitization of images in recent years has increased opportunities where an image acquired by an image input device such as a digital camera or a scanner is displayed with an image display apparatus such as a monitor or a projector, or printed with an image output apparatus such as an ink-jet printer or a laser printer.
Generally, in color reproduction techniques for reproducing colors of an object, an image input device acquires color information by separating original colors into three primary colors with filters of three colors, RGB. On the other hand, an image display device reproduces the original colors by performing an additive process using light emitters for three colors, RGB. An ink-jet printer and a laser printer reproduce the original colors by performing a subtractive process using colorants of four colors, CMYK.
Input/output of images is currently performed with various devices in various environments. For example, it is common for an image of an object to be acquired with a digital camera outdoors under sunlight, and the image printed out indoors to be viewed under a fluorescent lamp. Now, demands for faithfully reproducing original colors are increasing, especially in fields of professional use such as for camerapersons and designers. However, as described above, faithfully reproducing original colors in situations where the device/environment at the time of input is different from the device/environment at the time of output, requires strict management of color information.
Specifically, if different viewing illuminants are applied at the time of input and at the time of output, for example, if an image is taken under sunlight at a color temperature of about 5000 K and the printed image is viewed under an indoor electric lamp at a color temperature of about 3000 K, it is extremely difficult to faithfully reproduce original colors. The reason for this is as follows. In the above-described colorimetric color reproduction using the three primary colors, three tristimulus values including illuminant information (XYZ values or Lab values) are matched. Therefore, if different illuminants are applied at the time of input and at the time of output, a color matching relation is not established.
Accordingly, as a technique to faithfully reproduce original colors in various environments, a technique called spectral-based color reproduction has drawn attention in recent years. The spectral-based color reproduction is a technique of faithfully reproducing original colors of an object under any illuminant by matching the reflectance for each light wavelength (spectral reflectance) as well, rather than reproducing the colors of the object by matching the tristimulus values.
For example, according to Japanese Patent Laid-Open No. 2002-221931, an image input apparatus employing six types of filters is used to acquire a 6-channel image. The spectral reflectance of the object is estimated with Wienner estimation or the like and converted into a signal value of each of RGB channels of a display apparatus closest to the estimated spectral reflectance.
As another example, according to Japanese Patent Laid-Open No. 2000-333186, multi-channel filters are used to acquire a multi-band image of an object. The spectral reflectance of each pixel is computed and converted into RGB values of an output apparatus.
As an expansion of these techniques, an approach is also known in which apparatus such as a multi-band monitor, a multi-band projector, and a multi-color printer are used as output apparatus, and the spectral reflectance is converted into signal values for the respective output apparatus.
Typically, a band-pass filter that transmits only a particular band (wavelength range) is employed as a color filter used for each channel of an input apparatus. Representative types of band-pass filters include a gelatin filter, a pigment filter, a liquid crystal tunable filter, and an interference filter. Among these filters, the gelatin filter and the pigment filter are relatively inexpensive but transmit a wide bandwidth. The liquid crystal tunable filter and the interference filter are capable of narrowing the transmitted bandwidth, but they are costly compared with the gelatin filter and the pigment filter.
Here, the influence of the transmitted bandwidth of band-pass filters on the spectral sensitivity will be described. FIG. 1 shows examples of the spectral sensitivity of each channel when a multi-band input apparatus with six bands is constructed: an upper diagram illustrates a case where wide-bandwidth filters are used, and a lower diagram illustrates a case where narrow-bandwidth filters are used. With the wide bandwidth as shown in the upper diagram of FIG. 1, overlaps occur in transmitted band ranges of adjacent channels. The more channels the multi-band input apparatus has and the wider the bandwidth is, the wider the overlapping areas are. On the other hand, with the narrow bandwidth as shown in the lower diagram of FIG. 1, the overlaps of the spectral sensitivity between the channels decrease. However, since the amount of light transmitted by the filter for each channel decreases, an input image is susceptible to noise.
Therefore, associating input signals with output signals without involving spectrum information is contemplated. For example, Japanese Patent Laid-Open No. 2006-287585 discloses a technique in which a patch image output from an output apparatus is multi-band imaged, and a multi-dimensional look-up table (hereinafter referred to as LUT) that associates input signals with output signals of each pixel is generated. This allows converting input signals into output signals without involving the spectral reflectance as intermediate data, thereby reducing the occurrence of errors in the conversion processing. In addition, performing the LUT processing allows reducing the amount of computation required for the conversion processing.
A method is also known in which the spectral reflectance is computed with Winner estimation or the like for each grid point of the LUT and converted into a signal value of each channel of an output apparatus closest to the computed spectral reflectance.
As described in the above Japanese Patent Laid-Open No. 2002-221931 and Japanese Patent Laid-Open No. 2000-333186, the conventional spectral color reproduction techniques estimate the spectral reflectance of image data acquired with a multi-channel multi-band camera and then convert the spectral reflectance into signal values for an output apparatus. However, if, for example, the spectral reflectance data is sampled at 10 nm intervals in the range of 380 nm to 730 nm, 36-dimensional data must be handled for each color. This results in an enormous amount of CPU computation time and a huge memory size used, posing a problem of the difficulty to perform spectral color reproduction of natural images.
A technique described in Japanese Patent Laid-Open No. 2006-287585 uses a multi-dimensional LUT in which signal values of each channel of a multi-band input apparatus form an input value space. However, if wide-bandwidth filters are used in the input apparatus in order to address noise, the correlation between signals of adjacent channels becomes strong. Assuming that input signals are normalized in the range of 0 to 1, the input signals concentrate around a diagonal line connecting “a grid point where all signals are 0” and “a grid point where all signals are 1” in the multi-dimensional LUT. In other words, performing principal component analysis on the pixel distribution yields the first principal component around the diagonal direction of the LUT. This results in a drawback in that, although most of grid points apart from the diagonal line of the multi-dimensional LUT are not used for image processing, the data size of the LUT becomes huge. The excessively large data size of the LUT also has drawbacks such as a reduced speed of the LUT processing, or inability to obtain a sufficient grid point density for required accuracy. If the spectral reflectance of each grid point of the LUT is computed with Winner estimation or the like, the strong correlation between signals of adjacent channels causes reduced accuracy when the spectral reflectance is estimated based on input signal values that are out of the correlation. In this case, a reflectance smaller than 0 or greater than 1 may be computed.
Thus, in a multi-dimensional LUT in which input values form the input color space, the estimation accuracy tends to be lower for grid points farther from around the first principal component, that is, the diagonal line. Here, it is assumed that N-dimensional LUT conversion with a uniform grid point interval d on each axis is performed for an input signal value located at a distance x from the diagonal line. Then a grid point at a position distant by x+d√{square root over (N)} at the maximum in the direction orthogonal to the diagonal line is used. Therefore, there is a drawback in that performing the LUT processing starting from a grid point with relatively poor estimation accuracy of the spectral reflectance reduces the image conversion accuracy.
On the other hand, if a narrow-bandwidth filter is used for each channel of the input apparatus, an input image is susceptible to noise as described above. Therefore, again, there is a drawback of reduced image reproduction accuracy.
The present invention has been made for solving the above issues and provides an image processing apparatus and an image processing method in which an N-dimensional image (3<N) of a multi-band imaged object is color-separated with a high speed and accuracy and a small amount of memory use to enable color reproduction that is faithful to the object.